Maximum-Entropy Density Estimation for MRI Stochastic Surrogate Models

Stochastic Information Gradient Algorithm Based on Maximum Entropy Density Estimation

Maximum Entropy Density Estimation with Incomplete Data

We propose a natural generalization of Regularized Maximum Entropy Density Estimation (maxent) to handle input data with unknown values. While standard approaches to handling missing data usually involve estimating the actual unknown values, then using the estimated, complete data as input, our method avoids the two-step process and handles unknown values directly in the maximum entropy formula...

Maximum Entropy Models and Stochastic Optimality Theory

In a series of recent publications (most notably Boersma (1998); see also Boersma and Hayes (2001)), Paul Boersma has developed a stochastic generalization of standard Optimality Theory in the sense of Prince and Smolensky (1993). While a classical OT grammar maps a set of candidates to its optimal element (or elements), in Boersma’s Stochastic Optimality Theory (StOT for short) a grammar defin...

Estimation Bias in Maximum Entropy Models

Maximum entropy models have become popular statistical models in neuroscience and other areas in biology and can be useful tools for obtaining estimates of mutual information in biological systems. However, maximum entropy models fit to small data sets can be subject to sampling bias; i.e., the true entropy of the data can be severely underestimated. Here, we study the sampling properties of es...

Consistency and Generalization Bounds for Maximum Entropy Density Estimation

We investigate the statistical properties of maximum entropy density estimation, both for the complete data case and the incomplete data case. We show that under certain assumptions, the generalization error can be bounded in terms of the complexity of the underlying feature functions. This allows us to establish the universal consistency of maximum entropy density estimation.